Balancing with Noise and Delay 
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Motivated by recent studies in human balance control, we study a delayed random walk with an 
unstable fixed point. It is observed that the random walker moves away from the unstable fixed point 
more slowly than is observed in the absence of delay. It is shown that, for given a noise level, there 
exists an optimal delay to achieve the longest first passage time. Our observations support recent 
demonstrations that noise has a beneficial role for balance control and emphasize that predicitive 
'■^ , strategies are not necessary to transiently control balance. 
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INTRODUCTION 

Feedback delay and noise pose challenging problems for the neural control of balance. The time scales for the 
controlling motion in human postural sway Q and stick balancing at the fingertip 0, are much shorter than the 
1^ estimated latencies for the neural control mechanisms. How is it possible to control noisy, unstable systems whose 
f^i ■ time constant is much shorter than the control time delay? 

Paradoxically, noise has a beneficial effect for balancing tasks . Recently it has been shown that the fluctuations 
in the controlled variable for stick balancing at the fingertip, i.e. the vertical displacement angle 9 , exhibit on-off 
intermittency Q{: a -1/2-power law is present in the power spectrum and a -3/2 power law occurs for the laminar 
phases. Importantly, the time between > 98% of the successive controlling movements is less than the estimated neural 
latency. These phenomena can be reproduced by Q, i.e. a damped inverted pendulum stabilized by time delayed 
1 feedback with a noisy gain, provided that the parameters are chosen to place the system close to a stability boundary. 
Thus the mechanism for obtaining fast control in the presence of time-delayed feedback is related, at least in part, to 
O ■ the stochastic or chaotic forcing of important control parameters, e.g. gain, across a stability boundary. 

What is the mechanism for the stabilizing effects of the state-dependent noise? The stability boundary for ||TJ 
separates a stable fixed point from an unstable fixed point 0- It has been suggested that the beneficial effects of 
on-off intermittency for stick balancing arise because the fluctuations in 9 resemble a random walk for which the 
mean value of 9 is approximately zero, i.e. the balanced position is statistically stabilized Q- This interpretation 
suggests that the control parameters for stick balancing could be tuned to place the system either directly on the 
stability boundary or perhaps to the stable side of the boundary; the proximity to the stability boundary depending 
on the intensity of the state-dependent noise. Subsequently two experimental observations suggested that the system 
may actually be tuned slightly to the unstable side of the stability boundary: 1) power laws identical to those 
observed for stick balancing are observed in a virtual stick balancing task that does not possess a stable fixed point 
0; and 2) the observed survival function for stick balancing, i.e. the probability that a stick is still balanced at 
time t, is well described by a second-order stochastic discrete equation that possesses only an unstable fixed point 
P| . Taken together these observations lead to the surprising conclusion that an unstable fixed point can be stabilized 
in a dynamical system with retarded variables using state-dependent noise. The obvious question is does such a 
mechanism exist? 
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Here we study the first passage times of a delayed random walk in which the fixed point is unstable or repulsive. 
We find that in a repulsive delayed random walk the first passage time is prolonged for certain values of the delay. 
In other words, memory effects can transiently stabilize an unstable fixed point. These observations challenge the 
hypothesis that the neural control of balance solely involves the use of predictive control strategies @ . 

MODEL 

From a mathematical point of view, the study of stochastic dynamical systems with retarded variables leads naturally 
to formulations in terms of a stochastic delay-differential equation. In the case of stick balancing at the fingertip, it 
was proposed that the dynamics of the deviation angle 9 of the stick from the upright position during the human 
stick balancing task can be described by |2| 

m ^ 9{t) + 1 Jt 6[t) + K9{t) = {r ° + m)9{t ~ T) (1) 
where to, 7, K,ro are parameters with £(t) being the Gaussian white noise with strength D as follows, 

(£(*)> = 0, (2) 

mm) = D6(t- t >). (3) 

The time delay arises because of the neural latency in the initiation of the corrective movements. Depending on the 
choice of parameters the upright position can be stable or unstable. In order to reproduce the experimentally observed 
power laws it is necessary to chose the parameters close, or on, a stability boundary. 

The study of the behavior of in the unstable regime is not straightforward. Even today, the complexities involved 
in the study of stochastic delay equations in the stable regime impose formidable barriers 0, H, E|- In particular, 
the location of the stability boundary is uncertain if the intensity of either the state-independent or state-dependent 
noise is unknown |ldl . An alternate approach which is amenable to analysis is to approach the problem from the 
perspective of a delayed random walk ^2 0, ^| in which a walker takes a unit discrete step along one dimensional 
axis and the position of the random walker at time step t be given by X(t). The dynamics of the delayed random 
walk are governed by conditional probabilities which depend on the position of the walker at a time r in the past 

P(X(t + 1) = X(t) + l\X(t - t) > 0) = p (4) 

P(X(t + 1) = X(t) + l\X(t - r) = 0) - \ (5) 

P(X(t+l) =X(t) + l\X(t-r) <0) = 1-p, (6) 

where < p < 1 and r is the delay. In this class of model the delay corresponds to the neural delay and the conditional 
probabilities are determined by the stability of the fixed point. It should be noted that the fact that the conditional 
probabilities depend on the position of the walker they represent a form of state dependent noise. Such models 15] 
have already provided some insights into the nature of the fluctuations observed in human postural sway |l6| . 

For simplicity, set the fixed point at the origin, X = 0. With delay, the walker refers to its position in the past to 
decide on the bias, p, of the next step. Since the origin is repulsive we take p > 0.5. 

ANALYSIS AND SIMULATION 

In contrast to the attractive case, as the walker escapes away from the origin we do not have a stationary probability 
distribution. Hence, we need to look at the other statistical aspects to characterize its behavior. We focus here on 
the average first passage time and show that delay induces a peculiar behavior. 

We consider the average first passage time L to reach a certain position away from the origin, herein referred to 
as the limit point. In other words, we measure the average time for the walker starting from the origin to reach the 
limit point for the first time as we change parameters in the model. 

For the case of zero delay with the bias p, we can find an analytical result for this average first passage time L to 
reach the limit point ±X* as 

L = 2[-^-\ I I +— , (p ,6 0.5), (7) 

\q-pj \ i-£ I p-q 
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where we have set q = 1 — p. For the case of simple (symmetric) random walk with p = q = 0.5, this result reduces 
to an even simpler form. 

L = {X*f. (8) 

For the case of non-zero delay, such analytical result is yet to be obtained and computer simulation is used. We 
considered an ensemble of 10000 walkers. The initial condition is set so that the walker performs a normal random 
walk with no bias p = 0.5 for the duration of t — (— r, 0). The walker's position at t = is set as the origin X = 0. 
The limit point is set at ±X* . We measure the number of steps for each walker to go from the origin to ±X* and 
average them. We performed computer simulations for various bias p and delay r. 

Some sample results are shown in Figure ^ The most notable features of these graphs are the peaks in the graph, 
indicating that the longest first passage time appears at certain optimal values of r given bias p. In other words, the 
walker is most stabilized around the origin with appropriate non-zero delay. This is rather unexpected result contrary 
to the normal notion associated with effects of feedback delay, where longer delay increasingly de-stabilize systems. 
Here, appropriate combination of bias and delay time is inducing more stability. 




X X X X 



FIG. 1: Average first passage time L as we change r. The value of parameters (p,X*) are (a) (0.6,30), (b) (0.6,100), (c) 
(0.8,30), and (d) (0.8,100). 



In order to gain more insight into this phenomenon, we look for an approximate analytical expression, which is 
found to be given by the following expression 

L(t) = (1 + ar n e-^)L{T = 0). (9) 

Here a and f3 are parameters and r n is a normalized delay given as follows. 

p-q 



X* 



(10) 



This normalization uses a characteristic time dividing the distance X* by an average velocity of the walker p — q. 
Hence r n is a non-dimensionalized parameter as well. 




FIG. 2: Normalized average first passage time L n = g7~g(j) as we change normalized delay r«. The parameter sets (p,X*) 
plotted are (0.6,30), (0.6, 100), (0.8,30), and (0.8, 100) 

Figure shows this analytical approximation and the result of computer simulation. We see that the curves for 
various (p, X*) overlaps quite well with the analytical curve with appropriately chosen parameters of a — 1.27 and 
(3 = 0.67. We also notice that the peak height is approximately 1.7 times the average first passage time of zero delay 
case. 
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DISCUSSION 

A fundamental concept in the control of movement is that that nervous system calculates all of the forces required 
to produce a voluntary movement prior to its execution. Thus the control of movements on time scales faster than 
permitted by finite neural latencies is possible because all of the necessary plans have been made beforehand. However, 
even in well documented cases of predictive control such as stick balancing at the fingertip ]6j , th e survival times for 
stick balancing obey statisti cs p redicted for an unstable fixed point with noise and delay |5lll7j. Thus this theory 
is not without its problems 18]. Moreover emphasis on predictive control strategies seems contrary to work on the 
development of expertise suggesting that the ultimate goal of the nervous system is to minimize the need for active 
predictive control (for a recent review see [lflj ) . 

Our study is the first to show that the interaction between state-dependent noise and a time delay can transiently 
stabilize an unstable dynamical system. Obviously this form of stabilization does not involve predictive control, but 
depends on the relative values of the bias p and delay r. Recent studies of the development of expertise in stick 
balancing have also shown that with increased skill changes occur in both delay and the statistical properties of the 
controlling movements |l7j . Preliminary simulations indicate that the delayed random walk with repulsive origin 
reproduces the characteristics of the survival times observed for stick balancing (C. Luciani, T. Hosaka, T. Ohira, J. 
G. Milton, and J. L. Cabrera, unpublished observations). This observation is consistent with our suggestion that the 
control for stick balancing may be tuned to the unstable side of the stability boundary. 

However, the precise nature of this stabilizing mechanism is not yet clear. The observations in Figures ^ and 
El suggest that it could be related to a "resonance-like" phenomenon, such as that shown previously for delayed 
stochastic binary elements H3] ■ Another possibility is that it might be directly related to memory effects that 
arise through the influence of the initial function on the subsequent evolution of the dynamical system. 

The addition of noise can improve balance control in the elderly as measured from the fluctuations recorded on a 
force platform 4]. However, it is not yet known to what extent these noise- induced changes in fluctuations in the 
center of pressure translate into a reduced risk of falling in the elderly [il| • Could we do better if the effects of noise 
and delay on unstable dynamical systems were better understood? One possibility, suggested by our results, is that 
balance could be improved if the appropriate level of fluctuations could be introduced. There are various ways that 
this could be achieved. For example, as a test case, we have performed a preliminary experiment in which a subject 
is asked to shake an object with the other hand at the same time he balances a stick on his fingertip. In this way, 
fluctuations are meditated through a subject's body to his balancing fingertip. Though it takes some practices to find 
one's optimal wa y of generating a fluctuation, it is found that about 60% of the subjects can better balance the stick 
with fluctuation [22j. More experiments are required to explore such effects. 
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